Documentary relevance and structural hierarchy

THERE is no writer today more closely identified with research into the idea of relevance as operative within the field of documentation than DONALD J. HILLMAN. It is not certain that he sees any real hope for arrival at any sort of technique capable of producing documentsurrogates relevant to real queries-thus excepting those that satisfy queries derived from ‘source-documents’. Yet, in his paper “The Notion of Relevance (I)” [l] he cites, presumably because it is relevant to his own discussion, a paper by B. C. VICKERY, “The Structure of Retrieval Systems” [2]. No classifier, surely, probably not even an indexer, would have thought of using any such conceptual indicator as ‘relevance’ for Vickery’s paper. If (and it is really conditional) we agree with the proponents of citation-indices, it must be that VICKERY'S paper is relevant to HILLMAN'S topic. But why? Even more, how can a technique be devised capable of mechanically tracking down VICKERY'S paper as a result of the query ‘What is available on relevance ?’ ? HILLMAN puts the idea of relevance into question, but it seems to me that even so no one would be unable to say quite a bit about the idea, or at least to decide whether ‘document Z is relevant to query A’. The term itself has become fashionably popular since Cleverdon’s test-results were phrased in such a way as to emphasize relevance and recall above all other possible criteria of the evaluation of retrieval tests [3]. Yet it cannot be evaded that some things which someone thought (when he indexed a particular document) were relevant to certain topics were (from the querists’s point of view) not so;? it is this disparity which lowers the relevance ratio. Such disparity is also called (in conformity with another fashion, that of ‘information theory’) noise. Also, some things the querist would (presumably, if he were able to get at them) call relevant, but which the indexer did not, are missed, lowering the recall ratio.: This disparity could also be called (in conformity with the same fashion) silence. But another factor enters, which most reference librarians have encountered many times : purely conceptual relevance (match, fit between query and document-orientation) is not enough for a patron in the real world, even if it may be good enough in a test situation.